1. Field of the Invention
The present invention relates to color cameras, and particularly to a one-chip color camera including a color separation circuit for processing a signal from a solid state image sensing device with photoelectric conversion elements arranged in an array corresponding to color filters arranged in an array.
2. Description of the Background Art
In a color camera, the CCD (Charge Coupled Device) that is currently widely used as an image sensing device simply alters the amplitude of the output signal according to the brightness of the received light. Color information is not included in the output signal. It is therefore necessary to provide some optical means to obtain color information such as filtering the light incident on the CCD.
For color cameras of personal usage, the so-called one-chip system of extracting signals of the three, primary colors from a single CCD is employed. The so-called simultaneous color image sensing system using a color filter array at the light receiving side of the CCD is adopted.
[Structure of Inter-line Transfer CCD]
FIG. 24 is a schematic block diagram showing a structure of an inter-line transfer CCD 10 that is generally used as the CCD of a color camera for domestic usage.
Inter-line transfer CCD 10 includes a photosensitive unit 12 formed of pn junction type photodiodes arranged in an array, a transfer unit 14 including an analog register formed of CCDs, and a horizontal transfer register 16 receiving the charge from CCD analog shift register (transfer unit) 14 to transfer in the horizontal direction a signal corresponding to the sequentially transferred signal charge converted into voltage for output.
In FIG. 24, the pn junction type photodiodes are shown to correspond to three pixels arranged in both the vertical and horizontal directions for the sake of simplification. In practice, photodiodes corresponding to 500 pixels in the vertical direction and 500-800 pixels in the horizontal direction, for example, are arranged in an array for the CCD employed in a color camera.
The operation will be described briefly here.
When light enters a photodiode, charge is generated to be accumulated in the diode. Then, application of a predetermined voltage on a shift gate (well known but, for simplicity, not shown) causes all the accumulated charge to be transferred to analog shift register 14. CCD analog shift register 14 responds to application of clock pulse voltages xcfx86V1, xcfx86V2 and xcfx86V3 to transfer the charge sequentially towards horizontal transfer register 16. In horizontal transfer register 16, the received signal charge is converted into voltage to be sequentially output as an image sensed signal driven by externally applied horizontal drive signals xcfx86H1, xcfx86H2 and xcfx86H3.
[Driving Method of Inter-line Transfer CCD]
As the method of driving the inter-line transfer CCD, there are generally two modes, i.e., the frame accumulation system and the field accumulation system. Some color cameras employ the frame accumulation system that is based on an entire-pixel readout operation even for the color filter array of the color difference sequential system.
In the following, an inter-line transfer CCD operating in a frame accumulation mode that can read out for every pixel is contemplated.
[Structure of Conventional One-Chip Color Camera]
FIG. 25 is a schematic block diagram showing a structure of main components of a conventional one-chip color camera 2000.
One-chip color camera 2000 mainly includes an optical system 2 receiving light from an object of interest, a CCD 10 converting an optical image formed by optical system 2 into an electric signal, a drive circuit 2102 that drives to read out independently all the pixels for CCD 10, and a color separation circuit 2101 receiving the output signal from CCD 10 to output three primary color signals R, G and B corresponding to each pixel.
Color separation circuit 2101 includes a CLCRCB generation circuit 2104 that receives the signal output from CCD 10 to generate a luminance signal CL and color difference signals CR and CB, a random access memory 2106 (referred to as RAM hereinafter) to transfer a read out signal D (x, y), luminance signal CL, and color difference signals CR and CB, with CLCRCB generation circuit 2104, and that can retain these signals corresponding to at least one scanning line, and a matrix circuit 2108 that receives a signal from CLCRCB generation circuit 2104 to separate the three primary color signal RGB by a predetermined operation for output.
In one-chip color camera 2000 of FIG. 25, CCD 10 carries out a readout operation of all the pixels independently as mentioned above. More specifically, drive circuit 2102 generates a drive pulse that does not mix two pixels in the vertical direction in CCD 10.
[Method of Separating Color Difference Signal]
FIG. 26 is a schematic diagram of an arrangement of the color filter array for describing signal read out from CCD 10 including such a color filter array in a conventional one-chip color camera.
In the color difference sequential system shown in FIG. 26, color filters of magenta (represented as Mg hereinafter), green (represented as G hereinafter), cyan (represented as Cy hereinafter) and yellow (represented as Ye hereinafter) are arranged in a mosaic manner.
Since the so-called additive color process is applicable in the mixture of the color of light, the following relationship between the three primary colors of red (R), green (G) and blue (B) and the complementary colors of Mg, Ye and Cy is established.
Mg=R+Gxe2x80x83xe2x80x83(1)
Ye=R+Gxe2x80x83xe2x80x83(2)
Cy=B+Gxe2x80x83xe2x80x83(3)
By using the above Mg, G, Ye and Cy as the colors of the color filters, the intensity of the G signal having a great weight for the luminance signal out of the three primary colors of R, G and B can be set greater than that of the R and B signals.
[Generation of Pseudo Color signal]
As shown in FIG. 26, color filters Mg and G are arranged alternately in the horizontal direction (x direction) as to the 0-th line in the 0-th order in the vertical direction (y direction). Correspondingly, the signals of Mg and G are output alternately from CCD 10 as to the scanning line corresponding to this line. As to the next first line (corresponding to y=1), signals of Ye and Cy are output alternately.
Thus, the following relationship is established between each pixel (x, y) and the filter color in the CCD output signal when n is a natural number.
i) When y=4nxe2x88x923 or 4nxe2x88x921 (i.e., y=1, 3, 5 . . . ):
When x is an even number, the color filter is Ye.
When x is an odd number, the color filter is Cy.
ii) When y=4nxe2x88x924 (i.e., y=0, 4, 8, . . . ):
When x is an even number, the color filter is Mg.
When x is an odd number, the color filter is G.
iii) When y=4nxe2x88x922 (i.e., y=2, 6, 10, . . . ):
When x is an even number, the color filter is G.
When x is an odd number, the color filter is Mg.
For the purpose of convenience, the row corresponding to the color filters of Mg and G is an even numbered line, and the row corresponding to the color filters of Ye and Cy is an odd numbered line. However, the same applies for a color filter arrangement shifted by one line, for example, as will be apparent from the following description.
[Generation of Color Difference Signal from Four Pixels Fixed with Respect to Pixel (x, y)]
As mentioned above, signals are read out independently for all the pixels from CCD 10. These signals are retained in RAM 2106.
Therefore, a luminance signal CL (x, y), a first color difference signal CR (x, y) and a second color difference signal CB (x, y) for each pixel (x, y) can be generated by the following operation respectively.
In the following, the signal output from the photoelectric conversion element corresponding to a pixel (x, y) in CCD 10 is represented as D (x, y).
By the linear operation on the output signals from pixels (x, y), (x+1, y), (x, y+1), and (x+1, y+1) of two rows and two columns including pixel (x, y), a luminance signal and two color difference signals can be extracted.
In other words, the following representation is generally established.
Cc(x, y)=Kc(x, y)xc2x7D(x, y)+Kc(x+1, y)xc2x7D(x+1, y)+Kc(x, y+1)xc2x7D(x, y+1)+Kc(x+1, y+1)xc2x7D(x+1, y+1)xe2x80x83xe2x80x83(4)
(Here, c is any of L, R and B)
It is assumed that the following relationship is established between the color of the filter of pixel (X, Y) and coefficients KL (X, Y), KR (X, Y), KB (X, Y) to generate CLCRCB signals. Here, X=x, x+1, Y=y, y+1.
i) When the color filter of pixel (X, Y) is Cy:
KL(X, Y)=1, KR(X, Y)=xe2x88x921, KB(X, Y)=1
ii) When the color filter of pixel (X, Y) is Ye:
KL(X, Y)=1, KR(X, Y)=1, KB(X, Y)=xe2x88x921
iii) When the color filter of pixel (X, Y) is Mg:
KL(X, Y)=1, KR(X, Y)=1, KB(X, Y)=1
iv) When the color filter of pixel (X, Y) is G:
xe2x80x83KL(X, Y)=1, KR(X, Y)=xe2x88x921, KB(X, Y)=xe2x88x921
The generation process of a luminance signal and a color difference signal will be described more specifically hereinafter according to the relationship between the color of the filter and coefficients KL, KR and KB.
For example, a luminance signal and a color difference signal corresponding to pixel (0, 0) are obtained in FIG. 26.
Noting that equations (1)-(3) are established, luminance signal CL (0, 0) is represented by the following equation (5).   "AutoLeftMatch"                                                                                                              C                    L                                    ⁡                                      (                                          0                      ,                      0                                        )                                                  =                                ⁢                                                      D                    ⁡                                          (                                              0                        ,                        0                                            )                                                        +                                      D                    ⁡                                          (                                              1                        ,                        0                                            )                                                        +                                      D                    ⁡                                          (                                              0                        ,                        1                                            )                                                        +                                      D                    ⁡                                          (                                              1                        ,                        1                                            )                                                                                                                                              =                                ⁢                                  Mg                  +                  G                  +                  Ye                  +                  Cy                                                                                                        =                                ⁢                                                      (                                          R                      +                      B                                        )                                    +                  G                  +                                      (                                          R                      +                      G                                        )                                    +                                      (                                          B                      +                      G                                        )                                                                                                                          =                                ⁢                                                      2                    ⁢                    R                                    +                                      3                    ⁢                    G                                    +                                      2                    ⁢                    B                                                                                                            (          5          )                    
The first color difference signal CR (0, 0) is calculated from the following equation (6).   "AutoLeftMatch"                                                                                                              C                    R                                    ⁡                                      (                                          0                      ,                      0                                        )                                                  =                                ⁢                                                      D                    ⁡                                          (                                              0                        ,                        0                                            )                                                        +                                      D                    ⁡                                          (                                              0                        ,                        1                                            )                                                        -                                      D                    ⁡                                          (                                              1                        ,                        0                                            )                                                        -                                      D                    ⁡                                          (                                              1                        ,                        1                                            )                                                                                                                                              =                                ⁢                                  Mg                  +                  Ye                  -                  G                  -                  Cy                                                                                                        =                                ⁢                                                      (                                          R                      +                      B                                        )                                    +                                      (                                          R                      +                      G                                        )                                    -                  G                  -                                      (                                          B                      +                      G                                        )                                                                                                                          =                                ⁢                                                      2                    ⁢                    R                                    -                  G                                                                                          (          6          )                    
The second color difference signal CB (0, 0) is calculated by the following equation (7).   "AutoLeftMatch"                                                                                                              C                    B                                    ⁡                                      (                                          0                      ,                      0                                        )                                                  =                                ⁢                                                      D                    ⁡                                          (                                              0                        ,                        0                                            )                                                        +                                      D                    ⁡                                          (                                              1                        ,                        1                                            )                                                        -                                      D                    ⁡                                          (                                              0                        ,                        1                                            )                                                        -                                      D                    ⁡                                          (                                              1                        ,                        0                                            )                                                                                                                                              =                                ⁢                                  Mg                  +                  Cy                  -                  Ye                  -                  G                                                                                                        =                                ⁢                                                      (                                          R                      +                      B                                        )                                    +                                      (                                          B                      +                      G                                        )                                    -                                      (                                          R                      +                      G                                        )                                    -                  G                                                                                                        =                                ⁢                                                      2                    ⁢                    B                                    -                  G                                                                                          (          7          )                    
In other words, the luminance signal and the two color difference signals for pixel (0, 0) can be calculated using the output signals from pixels (0, 0), (1, 0), (0, 1), (1, 1).
The above example shows the case where pixel (0, 0) of interest corresponds to a color filter of magenta. As for a pixel of interest corresponding to a filter of another color, the luminance signal and the color difference signals can be obtained by a linear operation of the output signals from pixels (x, y), (x+1, y), (x, y+1), (x+1, y+1) of two rows and two columns including the relevant pixel (x, y).
[Problem in Generation of Color Difference Signal from Four Fixed Pixels]
The above color separation system imposes problems set forth in the following.
i) When luminance is modified in horizontal direction
Consider the case where there is a change H1 (an edge where the luminance level changes from left to right in the screen) of the luminance level in the horizontal direction as to the intensity of light incident on the CCD pixel array, as shown in FIG. 26.
For the sake of simplification, it is assumed that a light signal of achromatic color, that has no color component, is entered at the edge portion, and only the luminance level changes.
At this edge portion, the luminance level is modified by b/a (b greater than a) in the horizontal direction (x axis).
Calculation of a luminance signal and color difference signals for pixel (0, 0), for example, is set forth in the following.
For luminance signal CL (0, 0):   "AutoLeftMatch"                                                                                                              C                    L                                    ⁡                                      (                                          0                      ,                      0                                        )                                                  =                                ⁢                                                      D                    ⁡                                          (                                              0                        ,                        0                                            )                                                        +                                      D                    ⁡                                          (                                              1                        ,                        0                                            )                                                        +                                      D                    ⁡                                          (                                              0                        ,                        1                                            )                                                        +                                      D                    ⁡                                          (                                              1                        ,                        1                                            )                                                                                                                                              =                                ⁢                                                      a                    xc3x97                    Mg                                    +                                      b                    xc3x97                    G                                    +                                      a                    xc3x97                    Ye                                    +                                      b                    xc3x97                    Cy                                                                                                                          =                                ⁢                                                      a                    xc3x97                                          (                                              R                        +                        B                        +                        R                        +                        G                                            )                                                        +                                      b                    xc3x97                                          (                                                                        2                          ⁢                          G                                                +                        B                                            )                                                                                                                                (          8          )                    
For first color difference signal CR (0, 0):   "AutoLeftMatch"                                                                                                              C                    R                                    ⁡                                      (                                          0                      ,                      0                                        )                                                  =                                ⁢                                                      D                    ⁡                                          (                                              0                        ,                        0                                            )                                                        +                                      D                    ⁡                                          (                                              0                        ,                        1                                            )                                                        -                                      D                    ⁡                                          (                                              1                        ,                        0                                            )                                                        -                                      D                    ⁡                                          (                                              1                        ,                        1                                            )                                                                                                                                              =                                ⁢                                                      a                    xc3x97                    Mg                                    +                                      a                    xc3x97                    Ye                                    -                                      b                    xc3x97                    G                                    -                                      b                    xc3x97                    Cy                                                                                                                          =                                ⁢                                                      a                    xc3x97                                          (                                              R                        +                        B                        +                        R                        +                        G                                            )                                                        -                                      b                    xc3x97                                          (                                                                        2                          ⁢                          G                                                +                        B                                            )                                                                                                                                (          9          )                    
For second color difference signal CB (0, 0):   "AutoLeftMatch"                                                                                                              C                    B                                    ⁡                                      (                                          0                      ,                      0                                        )                                                  =                                ⁢                                                      D                    ⁡                                          (                                              0                        ,                        0                                            )                                                        +                                      D                    ⁡                                          (                                              1                        ,                        1                                            )                                                        -                                      D                    ⁡                                          (                                              0                        ,                        1                                            )                                                        -                                      D                    ⁡                                          (                                              1                        ,                        0                                            )                                                                                                                                              =                                ⁢                                                      a                    xc3x97                    Mg                                    +                                      b                    xc3x97                    Cy                                    -                                      a                    xc3x97                    Ye                                    -                                      b                    xc3x97                    G                                                                                                                          =                                ⁢                                                      a                    xc3x97                                          (                                              R                        +                        B                        -                        R                        -                        G                                            )                                                        +                                      b                    xc3x97                    B                                                                                                            (          10          )                    
Since it is assumed that there is no color component, the following two equations are established.
2Rxe2x88x92G=0
2Bxe2x88x92G=0
With 2R=2B=G=2S and R=S, B=S, G=2S inserted in the above equations (8)-(10), the following results are obtained for the first and second color difference signals.
CR(0, 0)=xe2x88x925S(bxe2x88x92a)xe2x80x83xe2x80x83(11)
CB(0, 0)=S(bxe2x88x92a)xe2x80x83xe2x80x83(12)
Therefore, a color difference signal will be generated for pixel (0, 0) that originally does not include a color component. As a result, pseudo color appears on the screen to become the cause of degrading the picture quality.
Similarly, when there is a change H2 in the luminance level in the horizontal direction shown in FIG. 2, the following is obtained by calculating the color difference signals for a pixel (1, 0).
CR(1, 0)=5S(bxe2x88x92a)xe2x80x83xe2x80x83(13)
CB(1, 0)=xe2x88x92S(bxe2x88x92a)xe2x80x83xe2x80x83(14)
Therefore, pseudo color is generated for pixel (1, 0) when there is a change H2 in the luminance level.
The same applies for all the pixels having an edge in the luminance level in the horizontal direction.
ii) When luminance changes in the vertical direction
When there is a change V1 in luminance in the vertical direction as shown in FIG. 26, calculation of the color difference signals for a pixel (0, 2) is set forth in the following.
For color difference signal CR (0, 2):   "AutoLeftMatch"                                                                                                              C                    R                                    ⁡                                      (                                          0                      ,                      2                                        )                                                  =                                ⁢                                                      D                    ⁡                                          (                                              1                        ,                        2                                            )                                                        +                                      D                    ⁡                                          (                                              0                        ,                        3                                            )                                                        -                                      D                    ⁡                                          (                                              0                        ,                        2                                            )                                                        -                                      D                    ⁡                                          (                                              1                        ,                        3                                            )                                                                                                                                              =                                ⁢                                                      b                    xc3x97                    Mg                                    +                                      a                    xc3x97                    Ye                                    -                                      b                    xc3x97                    G                                    -                                      a                    xc3x97                    Cy                                                                                                                          =                                ⁢                                                      a                    xc3x97                                          (                                              R                        +                        G                        -                        G                        -                        B                                            )                                                        +                                      b                    xc3x97                                          (                                              R                        +                        B                        -                        G                                            )                                                                                                                                (          15          )                    
For color difference signal CB (0, 2):   "AutoLeftMatch"                                                                                                              C                    B                                    ⁡                                      (                                          0                      ,                      2                                        )                                                  =                                ⁢                                                      D                    ⁡                                          (                                              1                        ,                        2                                            )                                                        +                                      D                    ⁡                                          (                                              1                        ,                        3                                            )                                                        -                                      D                    ⁡                                          (                                              0                        ,                        2                                            )                                                        -                                      D                    ⁡                                          (                                              0                        ,                        3                                            )                                                                                                                                              =                                ⁢                                                      b                    xc3x97                    Mg                                    +                                      a                    xc3x97                    Cy                                    -                                      b                    xc3x97                    G                                    -                                      a                    xc3x97                    Ye                                                                                                                          =                                ⁢                                                      a                    xc3x97                                          (                                              G                        +                        B                        -                        R                        -                        G                                            )                                                        +                                      b                    xc3x97                                          (                                              R                        +                        B                        -                        G                                            )                                                                                                                                (          16          )                    
By 2R=2B=G=2S, CR(0, 2)=0 and CB(0, 2)=0 are obtained, similarly.
It is therefore appreciated that no pseudo color is generated in the vertical direction.
Similarly, when there is a change V2 in the luminance signal in the vertical direction shown in FIG. 26, the values of the color difference signals both become 0 for the pixel at that edge portion. It is appreciated that a pseudo color is not generated for an edge in the vertical direction.
In the method of calculating a color difference signal according to output signals of pixels (x, y), (x+1, y), (x, y+1), (x+1, y+1) of two rows and two column including a pixel (x, y), a pseudo color is generated when there is a significant change (edge) in luminance in the horizontal direction. A pseudo color will appear at an area that originally has no color when an image is reproduced on the screen. As a result, the picture quality will be degraded.
Furthermore, in the process of generating signals R, G and B of the three primary colors from signals CL, CR, and CB, a pseudo luminance signal will be generated by the pseudo color signal component to cause degradation in the luminance signal.
An object of the present invention is to provide a one-chip color camera that can reproduce a luminance signal and a color difference signal at high resolution without degrading the resolution for a valid number of pixels.
Another object of the present invention is to provide a color separation method that allows generation of a corresponding luminance signal and corresponding color difference signals for each pixel according to output signals read out independently from the entire pixels of the CCD while suppressing generation of a pseudo color signal.
According to an aspect of the present invention, a one-chip digital color camera includes a solid state image sensing circuit and a color separation circuit.
The solid state image sensing circuit has a photoelectric conversion element corresponding to each pixel arranged in an array.
The solid state image sensing circuit includes a color filter array in which a color filter for a corresponding photoelectric conversion element are arranged in a mosaic manner.
The color filter array includes color filters of green and first, second and third complementary colors, respectively arranged corresponding to a group of pixels of arbitrary (2xc3x97m) rows and (2xc3x97n) columns (m, n: natural number) in the solid state image sensing circuit.
The color separation circuit generates color data corresponding to the center position of the pixels in the (2xc3x97m) rows and (2xc3x97n) columns according to the output from the solid state image sensing circuit. The color separation circuit includes a color interpolation circuit and a separation operation circuit.
The color interpolation circuit functions as a low-pass filter for an output from a photoelectric conversion element arranged with a corresponding color filter out of the outputs from a plurality of photoelectric conversion elements corresponding to the pixels of (2xc3x97m) rows and (2xc3x97n) columns in at least one of the column direction and the row direction for each of green color and first, second and third complementary colors. The separation operation circuit receives the output from the color interpolation circuit to separate the color data corresponding to the center position.
According to another aspect of the present invention, a method is provided of isolating color data according to a signal read out from a solid state image sensing device. The solid state image sensing device includes a color filter array of m first rows and m second rows arranged alternately for each one row. Each first row includes a green color filter and a first complementary color filter arranged alternately. Each second row includes a second complementary color filter and a third complementary color filter arranged alternately. The method of separating color data includes the step of carrying out a process as a low-pass filter for an output from a photoelectric conversion element arranged with a corresponding color filter out of the outputs from a plurality of photoelectric conversion elements corresponding to the pixels of (2xc3x97m) rows and (2xc3x97n) columns in at least one of the column direction and the row direction for each color of green and the first, second and third complementary colors, and the step of receiving a signal processed by the low-pass filter to separate color data corresponding to the center position of the photoelectric conversion elements of (2xc3x97m) rows and (2xc3x97n) columns.
The main advantage of the present invention is that generation of a pseudo color signal can be suppressed even when there is an edge of a sudden change in the luminance level by a change in the brightness of an object of interest, while maintaining high resolution for a luminance signal. Therefore, an image of high quality with no degradation in the picture can be reproduced.
The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.